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Abstract 

We  describe  a  new  architecture  to  integrate  a  psychological  model  into  a  crowd  simulation  system  in  order  to 
obtain  believable  emergent  behaviors.  Our  existing  crowd  simulation  system  (MACES)  performs  high  level 
wayfinding  to  explore  unknown  environments  and  obtain  a  cognitive  map  for  navigation  purposes,  in  addition  to 
dealing  with  low  level  motion  within  each  room  based  on  social  forces.  Communication  and  roles  are  added  to 
achieve  individualistic  behaviors  and  a  realistic  way  to  spread  information  about  the  environment.  To  expand  the 
range  of  realistic  human  behaviors,  we  use  a  system  (PMFserv)  that  implements  human  behavior  models  from  a 
range  of  ability,  stress,  emotion,  decision  theoretic  and  motivation  sources.  An  architecture  is  proposed  that 
combines  and  integrates  MACES  and  PMFserv  to  add  validated  agent  behaviors  to  crowd  simulations. 
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1.  Introduction 


There  are  many  applications  of  computer  animation  and  simulation  where  it  is  necessary  to 
model  virtual  crowds  of  autonomous  agents.  Some  of  those  applications  include  education, 
entertainment,  training  (for  the  military  and  police)  and  human  factors  analysis  for  building 
evacuation  or  other  scenarios  where  masses  of  people  gather  such  as  sport  events  and 
concerts.  Many  producers  and  consumers  of  training  simulator  and  game  environments  are 
beginning  to  envision  a  new  era  where  psycho-socio-physio-logical  models  could  be 
intertwined  to  enhance  their  environments'  simulation  of  human  agents. 

Animating  virtual  crowds  is  often  mediated  by  local  rules  [17],  forces  [11],  or  flows  [3].  The 
next  challenge  in  crowd  animation  is  to  simulate  realistically  how  communication  affects  the 
behavior  of  autonomous  agents  and  how  having  a  psychological  model  for  each  agent  can 
enhance  the  emergent  behavior  of  the  crowd  [24,25]. 

Most  crowd  simulation  systems  are  implemented  by  having  a  large  number  of  individuals 
who  have  exactly  the  same  behavior.  Some  systems  offer  a  limited  variety  of  behaviors  by 
differentiating  agents  based  on  age  and  gender.  Current  crowd  simulation  systems  lack  a  wide 
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variety  of  behaviors  based  on  demographically  and  culturally  validated  behaviors.  In  order  to 
achieve  this  goal,  we  believe  there  is  a  need  for  a  psychological  model  driving  each  agent’s 
mind  and  therefore  its  decision  making  process. 

When  simulating  human  behavior,  it  is  essential  to  model  the  psychology  factors  that  affect 
their  decisions.  For  example  when  simulating  panic  and  time  pressure  during  evacuation,  it  is 
important  to  model  the  following  observations: 

•  Individuals  may  not  be  aware  of  the  internal  connectivity  of  the  building  and  therefore 
may  ignore  some  suitable  paths  for  evacuation  [23]. 

•  Rising  stress  levels  have  the  effect  of  diminishing  the  full  functioning  of  one’s  senses, 
which  leads  to  a  general  reduction  of  awareness,  especially  the  ability  to  orient  oneself 
quickly  in  rooms  and  surrounding  areas  [29]. 

•  Agents  that  have  not  been  properly  trained  are  likely  to  feel  stressed  and  might  reach 
the  point  where  they  find  themselves  incapable  of  making  the  right  decision  due  to 
time  pressure  [9],  On  the  other  hand,  trained  individuals  such  as  firefighters  deal  better 
with  a  dynamically  changing  environment  and  choose  the  best  sequence  of  actions 
based  on  their  perception  and  knowledge  of  the  environment. 

To  take  into  account  psychological  elements  that  affect  human  behavior,  we  make  use  of 
PMFServ  [24,25].  PMFserv  was  conceived  as  a  software  system  that  would  expose  a  large 
library  of  well-established  and  data-grounded  Performance  Moderator  Functions  (PMFs)  and 
Human  Behavior  Representations  for  use  by  cognitive  architectures  deployed  in  a  variety  of 
simulation  environments.  Its  principal  feature  is  a  model  of  decision-making  based  on 
emotional  subjective  utility  constrained  by  stress  and  physiology.  PMFserv  has  become  an 
agent  architecture  in  its  own  right  -  with  the  flexibility  to  either  act  as  a  meta-level  emotional 
arbitrator  for  other  cognitive  architectures  or  provide  a  fully  functional  stand-alone  system  to 
simulate  human  decision  making. 

The  idea  of  using  a  psychological  model  is  that  agents  will  operate  independently  in 
perceiving  the  simulated  world  and  in  forming  their  reactions  to  it.  At  no  point  will  they  be 
pre-scripted  or  programmed  via  rules  or  procedures.  We  only  model  personal  value  weights 
on  a  need  hierarchy  as  well  as  cultural  standards,  and  individual  agents  will  make  their  own 
(micro)decisions  that  lead  to  the  emergent  macro-behavior. 

In  this  paper  we  present  a  framework  that  combines  the  PMFserv  psychological  model  in  our 
MACES  crowd  simulation  system  [19].  MACES  deals  not  only  with  the  local  motion  of 
individuals  within  a  room  but  also  with  the  wayfinding  process  that  allows  them  to  explore 
and  learn  the  internal  structure  of  a  building.  Therefore  our  agents  can  find  their  way  around 
an  environment  that  they  were  not  familiar  with  in  advance.  Agents  can  also  communicate 
among  themselves  to  share  infonnation  and  manifest  varying  behaviors  based  on  their 
different  roles. 

2.  Related  Work 


There  have  been  several  cognitive  agent  architectures  proposed  to  generate  human-like 
behaviors.  Brogan  and  Hodgins  [2]  used  particle  systems  and  dynamics  for  modeling  the 
motion  of  groups  with  physics.  Helbing  [11]  describes  methods  to  simulate  the  movement  of 
pedestrians  based  on  a  social  force  model  which  is  a  microscopic  (personal)  approach  for 
simulating  pedestrian  motion.  It  solves  Newton’s  equation  for  each  individual  and  considers 
repulsive  interactions,  friction  forces,  dissipation  and  fluctuations.  Tu  and  Terzopoulos 


worked  on  behavioral  animation  for  creating  artificial  life,  where  virtual  agents  are  endowed 
with  synthetic  vision  and  perception  of  the  environment  [8].  Reynolds  [21]  first  used  a 
distributed  behavioral  model  to  produce  flocking  behavior. 

Traditional  crowd  simulators  ignore  the  differences  between  individuals  and  treat  everyone  as 
having  the  same  simple  behavior,  but  there  are  other  models  that  represent  each  individual  as 
being  controlled  by  rules  based  on  physical  laws  or  behavioral  models.  In  a  multi-agent 
system,  the  agents  are  autonomous  and  typically  heterogeneous.  Research  here  is  concerned 
with  coordinating  intelligent  behaviors  among  a  collection  of  autonomous  agents  so  that  they 
can  coordinate  their  knowledge,  goals,  skills,  and  plans  jointly  to  take  action  and  to  solve 
problems.  Some  applications  include  crowd  behavioral  models  used  in  military  training  [30] 
and  simulations  to  support  architectural  design  both  for  everyday  use  [  1  ]  [22]  and  for 
emergency  evacuation  conditions  [26,  17]. 

Other  models  have  been  incorporated  into  commercial  software  tools  such  as  regression  [16], 
route-choice  [12],  queuing  [14],  gas-kinetic  [10],  and  microscopic  models.  Of  particular 
relevance  to  individualized  actions,  the  microscopic  models  describe  the  time-space  behavior 
of  individual  agents.  There  are  two  subcategories:  the  Social  Force  models  and  the  Cellular 
Automata  (CA)  models.  The  difference  between  them  is  in  the  discretization  of  space  and 
time.  Social  Force  models  [11]  describe  agent  behavior  microscopically  by  social  fields 
induced  by  the  social  behavior  of  the  individuals.  In  the  Cellular  Automata  approach  the  study 
area  is  represented  by  a  uniform  grid  of  cells  with  local  states  depending  on  a  set  of  rules 
which  describe  the  behavior  of  the  cell  occupants  [3,13].  These  rules  compute  the  state  of  a 
particular  cell  as  a  function  of  its  previous  state  and  the  states  of  the  adjacent  cells. 

In  order  to  reduce  the  complexity  of  controlling  all  the  agents  in  the  crowd,  while  still 
maintaining  detailed  behaviors,  several  systems  have  attached  information  to  the  environment 
[7,28,27].  Our  MACES  system  also  embeds  information  in  the  environment,  such  as  shortest 
paths.  Individual  agents  will  have  differential  types  of  access  to  that  information  and  they  will 
use  it  in  different  ways  depending  on  their  individual  behavior  at  any  given  moment. 

The  PMFServ  psychological  model  was  not  designed  for  a  specific  problem  domain  or  level 
of  detail,  and  as  a  result  offers  a  high  degree  of  flexibility  of  representation  that  allows  it  to  be 
readily  employed  in  many  simulation  domains.  It  has,  however,  been  used  to  build 
physiological  and  psychological  models  of  crowd  members.  Figure  1  shows  a  screen  capture 
of  a  protest  scene  developed  using  PMFServ  and  Opensteer  (http://opensteer.sourceforge.net/) 


Figure  1:  Screen  capture  of  a  protest  scene  developed  in  using  PMFServ  and  Opensteer. 


3 .  Architecture  Overview 


Presently  we  are  designing  and  implementing  an  integrated  architecture  to  generate  realistic 
crowd  simulation  with  a  wide  variety  of  individualistic  behaviors.  We  first  describe  the 
MACES  system  [19]  which  implements  and  illustrates  the  agent  bodies,  actions  and  results 
when  using  communication  and  roles,  and  then  the  PMFserv  system  which  provides  the 
agent’s  motivations,  stress,  copying  style,  emotions,  personality  and  decisions.  Finally  we  will 
describe  how  we  can  integrate  these  two  systems  in  order  to  enhance  the  emergent  behavior  of 
crowd  simulation  by  applying  psychosocial  behavioral  moderators  to  each  of  the  individuals. 


3.1.  MACES :  Multi- Agent  Communication  for  Evacuation  Simulation 

MACES  computes  agent  navigation  at  two  levels.  The  high  level  corresponds  to  the 
wayfinding  process  of  finding  the  sequence  of  rooms  towards  an  exit,  and  the  low  level 
corresponds  to  the  local  movement  within  each  room  using  social  forces.  For  a  given 
environment,  the  shortest  paths  from  each  room  towards  an  exit  will  be  saved  within  the 
environment.  This  infonnation  can  represent  either  the  known  path  that  an  individual  would 
have  followed  when  entering  the  building,  or  the  path  indicated  by  the  emergency  exits. 

Each  agent  will  have  its  own  mental  map  which  abstracts  the  geometry  of  the  building  and  is 
represented  by  a  cell  and  portal  graph,  where  the  nodes  are  the  rooms  and  the  arcs  are  the 
portals  between  rooms.  Mental  maps  will  be  expanded  as  an  agent  explores  the  environment 
and  shares  information  with  other  individuals  of  the  crowd  through  the  communication 
process. 

The  communication  process  occurs  between  individuals  in  a  room.  It  involves  information 
about  location  of  hazards  blocking  possible  paths  and  directions  within  a  room  that  have  been 
explored  and  where  no  exit  was  found.  This  localized  sharing  of  mental  models  is  the  key  to 
our  algorithm’s  wayfinding  behavior. 

Different  roles  are  applied  to  simulate  a  variety  of  behaviors.  These  roles  depend  on  two 
attributes:  leadership  and  training: 

•  Trained  leaders  have  complete  knowledge  about  the  building’s  internal  structure  and 
would  help  others  during  the  evacuation  process.  An  example  of  this  type  of  agent 
would  be  a  firefighter. 

•  Untrained  leaders  correspond  to  people  that  by  nature  can  handle  stress  better,  tend  to 
help  others  and  will  explore  the  building  searching  for  new  paths. 

•  Untrained  non-leaders  (followers)  represent  dependent  people  who  might  panic  during 
an  emergency  situation  and  reach  the  point  where  they  are  incapable  of  making  their 
own  decisions. 

The  results  obtained  using  MACES  for  crowd  evacuation  show  significant  improvements  in 
evacuation  rates  when  using  communication,  the  different  emergent  behaviors  obtained  by 
applying  roles  to  the  individuals  of  the  crowd,  and  the  relevance  of  having  trained  people  in 
the  crowd  during  an  emergency  evacuation.  From  the  experimental  data,  we  have  observed 
that  only  a  relatively  small  percentage  of  trained  (building  plan  knowledgeable)  leaders  yields 
evacuation  rates  comparable  to  the  case  in  which  everyone  is  trained. 


As  an  example,  Figure  2  shows  the  different  emergent  behaviors  obtained  when  using  a  high 
percentage  of  leaders  vs.  low  percentage.  The  image  on  the  left  shows  the  high  percentage  of 
leaders  unfamiliar  with  the  environment  who  tend  to  explore  it  looking  for  exits.  As  a  result, 
we  can  observe  small  groups  of  people  searching  the  space  and  sharing  information  which 
yields  a  faster  evacuation  time.  The  image  on  the  right  shows  a  small  percentage  of  leaders 
and  therefore  a  large  number  of  followers.  As  a  result,  the  emergent  behavior  consists  of  a 
few  large  groups  of  people  wandering  together,  following  others  and  unable  to  make  their 
own  decisions,  yielding  an  overall  longer  evacuation  time. 


Figure  2:  Example  picture  sequence  of  crowd  evacuation  with  high  vs.  low  percentage  of  leaders 


3.1.1.  High  Level:  Wayfinding 

The  high  level  component  of  MACES  involves  finding  a  path  towards  an  exit.  In  order  to 
achieve  this,  agents  will  perform  different  actions  depending  on  their  roles  that  will  allow 
them  to  explore  and  learn  the  features  of  the  environment.  Figure  3  shows  the  three  main 
steps  of  the  high  level  algorithm. 


Figure  3:  High  Level  Wayfinding  diagram 


The  first  step  is  the  communication  process.  Agents  within  a  room  share  information 
(symbolically)  about  the  environment  which  consists  of  hazards  found  and  directions  to  avoid 
because  they  have  already  been  fully  explored  and  no  exit  was  found;  e.g.  “Do  not  go  through 
door  X,  there  is  no  exit”,  or  “Fire  found  going  north”.  With  the  information  received,  each 
agent  updates  its  mental  map  of  the  environment.  In  the  second  step  each  agent  checks  its 
updated  mental  map  to  see  whether  the  shortest  path  is  blocked.  Finally  the  third  step  consists 
of  finding  a  new  evacuation  path  when  the  known  one  is  found  to  be  blocked.  Agents  will 
proceed  with  different  behaviors  depending  on  their  roles.  Trained  leaders  with  complete 
knowledge  of  the  internal  structure  of  the  environment  will  just  follow  the  next  shortest  path 
known.  Untrained  leaders  need  to  explore  the  environment  in  order  to  find  a  new  evacuation 
route.  In  order  to  achieve  this,  they  perform  an  iterative  depth  first  search.  The  last  agent  type 
are  followers,  which  basically  wait  for  others  to  make  a  decision  and  then  they  just  follow 
them. 

3.1.2.  Low  Level:  Local  Motion 

The  local  motion  within  each  room  is  based  on  Helbing’s  model  [11]  which  describes  human 
crowd  behavior  with  a  mixture  of  socio-psychological  and  physical  forces: 

Pedestrians  l<i<N  of  mass  in;  like  to  move  with  a  certain  desired  speed  v,°  in  a  certain 
direction  ei°  and  they  tend  to  adapt  their  instantaneous  velocity  v,  within  a  certain  time 
interval  x;.  At  the  same  time,  the  individuals  try  to  keep  a  distance  from  other  individuals  j  and 
from  the  walls  w  using  interaction  forces  f]j  and  fw.  The  change  of  velocity  in  time  t  is  given 
by  the  acceleration  equation: 


rrij 


dv, 

dr 


=  m,- 


v”(r)c“(r)  -  Vj(r) 


Xf*+  XiiW 
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This  model  generates  realistic  phenomena  such  as  arching  in  the  portals  [20]  and  the  “faster  is 
slower”  effect.  [6].  In  our  model  (Figure  4)  the  desired  velocity  direction  within  each  room  is 
given  by  an  attractor  point  that  is  located  close  to  the  next  portal  that  the  agent  needs  to  cross. 
Therefore,  each  agent  will  walk  towards  the  attractor  while  avoiding  collision  with  walls  and 
with  other  agents. 


Figure  4:  Arch  formation  and  attraction  points 


3.2.  PMFServ 

Although  the  MACES  emergent  behaviors  are  significant,  individual  agents  are  limited  to 
three  distinct  roles.  For  more  realistic  human  behavior  we  need  to  expand  the  psychological 
representation.  PMFserv  ideally  suits  this  role.  PMFserv  agents  are  built  around  a  working 
memory  data  structure  loosely  corresponding  to  a  short-term  memory  system.  Modular  PMF 
subsystems  manipulate  data  contained  both  in  the  working  memory  and  in  a  long-term 


memory  store.  Information  is  layered  on  the  working  memory  such  that  each  layer  is 
dependent  on  the  layers  below  it  for  a  given  decision  cycle  of  the  agent  (see  Figure  5). 
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Figure  5:  PMFserv  Overview 


Observing  the  working  memory  from  the  bottom  up  reveals  the  decision  cycle  of  a  single 
agent.  Physiological  data  across  a  range  of  measures  (including  arousal,  fatigue,  hunger, 
thirst,  injury,  etc)  are  combined  to  set  the  levels  of  a  series  of  stress  reservoirs.  The  stress 
reservoirs  then  determine  the  agent’s  coping  style  (a  measure  of  the  agent’s  current  awareness 
and  capacity  for  rational  thought)  for  the  current  decision  cycle.  Need  reservoirs 
corresponding  to  the  degree  to  which  the  agent  has  satisfied  the  needs  outlined  by  Maslow 
[15]  are  set  based  on  any  action  that  might  have  occurred  in  between  decision  cycles. 

Based  on  the  agent’s  coping  style,  physiology,  and  any  memory  elements  that  might  have 
been  created  prior  to  the  current  cycle,  each  object  in  the  system  executes  its  perception  rules 
to  determine  which  objects  are  currently  perceivable  by  the  agent  and  generates  a  list  of  how 
these  perceptions  affect  the  utility  of  actions  in  the  environment.  These  perceptions  of 
available  actions  are  called  affordances  [4]. 

These  affordances  are  represented  in  terms  of  the  agent’s  emotion  model.  Our  emotion  model 
is  based  on  the  OCC  model  [18].  The  general  idea  is  that  an  agent  possesses  three  hierarchical 
trees  that  describe  the  agent’s  Goals  for  Action,  Standards  for  Behavior,  and  Preferences 
(GSR  trees)  for  People,  Objects,  and  Situations,  respectively.  An  action  can  be  represented  by 
a  series  of  successes  and  failures  on  the  sub-nodes  of  these  three  trees.  Each  sub-goal  is  given 
a  weight  that  describes  how  much  it  contributes  to  its  parent  node.  To  determine  the 
emotional  utility  of  an  action,  the  OCC  model  multiplies  the  degree  of  success  and  failure  of 
each  node  up  the  trees.  From  the  top  nodes  on  each  tree,  1 1  pairs  of  oppositely  valenced 
emotions  are  generated.  By  summing  those  emotions  we  arrive  at  a  utility  value  for  the  action 
under  consideration.  This  process  is  completed  for  every  afforded  action  available  to  the 
agent.  The  action  with  the  highest  utility  value  is  chosen  and  executed. 

4.  Interface  between  MACES  and  PMFserv 


Our  first  major  goal  is  to  improve  the  MACES  crowd  simulation  system  by  including  a 
PMFserv  psychological  model  that  can  affect  the  decision  making  process  to  achieve  a  richer 
variety  of  individualistic  behaviors  and  more  realistic  emergent  crowd  behaviors.  The 


psychological  model  will  affect  not  only  the  low  level  motion  (i.e.  modifying  the  speed  of 
actions  depending  on  the  psychological  state  of  the  individual)  but  also  the  high  level 
wayfinding  (i.e.  high  levels  of  stress  lead  to  loss  of  orientation). 

MACES  implements  and  illustrates  the  agent  bodies,  actions  and  results  while  PMFserv 
provides  the  agent’s  motivations,  stress,  coping  style,  emotions,  personality  and  decisions. 
PMFserv  implements  the  decision  making  processes  while  MACES  manages  the  situated 
planning  processes  (wayfinding  and  local  motion). 

For  each  agent,  PMFserv  would  operate  its  perception  and  run  its  cognition  to  determine 
collective  and  individual  action  decisions  and  pass  instructions  back  to  MACES  to  carry  out 
the  resulting  actions  and  emergent  behaviors.  Figure  6  highlights  the  main  interaction  loop. 


Figure  6:  Main  interface  between  MACES  and  PMFserv 


MACES  needs  to  have  as  an  input  the  high  level  goals  or  actions  that  need  to  be  performed 
such  as  “find  the  exit”  and  the  emotional  state  of  each  agent.  MACES  will  execute  these 
actions  based  on  current  knowledge,  the  events  perceived  from  the  environment  and  its 
current  emotional  state.  In  PMFserv  each  agent  is  guided  by  the  three  value  trees  offering  a 
goal  hierarchy,  standards  (based  on  for  example  ethics,  law,  etc)  and  preferences. 

MACES  will  provide  information  about  the  environment  such  as  hazards  that  have  been  seen 
or  information  obtained  though  communication  with  other  agents.  PMFserv  will  then  modify 
the  GSP  trees  and  therefore  the  emotional  state  of  each  agent.  Table  1  shows  a  summary  of 
the  main  modules  and  functionalities  that  each  system  offers. 


PMFserv 

MACES 

-  Event  sensor 

-  Psychology/Stress  Module 

-  Personality,  Culture  and  Emotion  Module 

-  Decision  Module 

-  Response  Selector 

-  Semantic  labeling  of  Events 

-  World  object  Affordances 

-  Roles 

-  Communication  Module 

-  Local  motion  (obstacle  avoidance,  herding,  etc.) 

-  Memory  (mental  maps) 

-  Display  services  (2D  and  3D,  both  real  time) 

-  Animation 

-  Wayfinding  Module 

Table  1:  Main  modules  and  functionalities  of  each  system. 


MACES  can  interact  with  PMFserv  via  an  API  that  allows  the  external  simulator  to  change 
properties  of  objects  in  PMFserv  and  to  generate  events  that  an  agent  can  perceive,  resulting 
in  responsive,  reactive,  and  situated  behaviors. 


5.  Conclusions 


We  have  presented  a  framework  to  combine  a  psychological  model  (PMFserv)  with  a  crowd 
simulation  system  (MACES).  MACES  offers:  (a)  a  high  level  wayfinding  algorithm  to  allow 
individuals  in  a  crowd  to  explore  an  unfamiliar  building  in  order  to  find  exits  during  an 
emergency  (b)  inter-agent  communication  to  share  knowledge  of  the  building  during  high 
level  wayfinding,  and  (c)  inclusion  of  certain  roles  to  embed  individualism  in  the  crowd. 
PMFserv  offers  mature  models  for  physiology,  stress,  perception  and  emotion.  PMFserv  can 
handle,  e.g.,  the  dysfunctional  behavior  that  emerges  in  people  during  disasters,  such  as 
trance-like  disbelief,  milling,  grouping  and  docile  sheep-like  following. 

With  the  framework  we  presented,  we  are  working  towards  developing  a  crowd  simulation 
system  able  to  achieve  a  wide  variety  of  emergent  behaviors  based  on  validated  human 
psychological  factors. 
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